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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)E3 Responsive to communication(s) filed on 13 February 2006 . 
2a)EI This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 7-20 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 7-20 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 
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3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
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DETAILED ACTION 

1 . This action is responsive to communications of amendment received 2/13/2006. 

2. The disposition of the claims is as follows: claims 7-20 are pending in the application. 
Claim 7 is the independent claim. Claims 1 - 6 have been withdrawn. 

Election/Restrictions 

3. This application contains claims 1-6 drawn to an invention nonelected with traverse in 
Paper dated 1 1/29/2005. A complete reply to the final rejection must include cancellation of 
nonelected claims or other appropriate action (37 CFR 1.144) See MPEP § 821.01. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 7, 8 and 10-12 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
"Affective Wearables" by Picard and Healey, hereinafter Picard, and further in view of Hassan et 
al., (US 5,550,646), hereinafter Hassan. 

A. Picard discloses claim 7, "A method for providing a retrieval scheme for stored digital 
images, [Picard - p. 91, in Applications of affective wearables at 'Augmenting a system like 
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Steve Mann's Wearcam [Man97) with affective sensing and pattern recognition could help it 
learn when to remember' the video it collects, as opposed to always relying on the user to tell it 
what to remember or forget. Suppose for example that you let the camera roll while playing with 
a cute little baby. It might notice that you always save the shots when the baby makes you laugh, 
or smile. By detecting these events, it could become smarter about automatically saving these 
photos. Moreover, by labeling the photos with these affective events, you can later ask for ones 
by their affective qualities, "Computer, please show us the funny images.'"] comprising the steps 
of: 

a) electronically storing a plurality of digital images [corresponds with ' save the shots ' , 
supra in Picard]; 

b) providing a user identifier which identifies a particular user for the plurality of digital 
images; 

c) classifying one or more of the images as an important image based upon the particular 
user's reaction to the images [corresponds with Picard, supra, at 'it could become smarter about 
automatically saving these photos. Moreover, by labeling the photos with these affective events, 
you can later ask for ones by their affective qualities , "Computer, please show us the funny 
images .'" 

Furthermore in Picard, p. 93, at 'In psychological studies by Lang [LGeaSS] and Winton, 
Putnam and Krauss [WPK84], subjects were monitored while looking at a series of photographs 
which were supposed to elicit an emotional response . These studies showed that heart rate 
variability was an indication of valence (whether or not the person found the photograph 
pleasant) and that the ratio of skin conductance to heart rate variability was an indicator of 
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arousal. However, the greatest changes in skin conductance and heart rate in their study were 0.6 
micro-Siemens, and 8 beats per minute respectively, in the ten second period after viewing the 
slides.']; and 

d) electronically storing the user identifier and the classification to facilitate retrieval of 
particular ones of the plurality of stored digital images by the particular user" supra [as detail]. 

However Picard does not appear to disclose, "providing a user identifier which identifies 
a particular user for the plurality of digital images" and "electronically storing the user 
identifier", but Hassan does in col. 4, Ins. 43-64 - see 'For example, when an image is captured, 
the user of the system may be prompted to enter a supplemental ID number or other text 
information by displaying a legend on LCD display 215 that reads "enter ID on keypad". This 
supplemental ID or relevant notes would be entered by a user via alphanumeric keypad 21 1, and 
stored with the digital image in RAM 207. The supplemental ID or notes could later be recalled 
and displayed on LCD display 215, so that a user could be reminded of important facts (such as 
client name, file number, etc.) associated with a particular digital image.' 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply affective information to saving images disclosed by Picard in 
combination with supplemental ID stored with the digital image disclosed by Hassan, and 
motivated to combine the teachings because they both employ recording digital images with ID 
or labeling as revealed by Hassan in col. 1, lines 47-67 and by Picard using WearCam. 

Although the funny images were not identified by the specific term "important", they 
(funny images) were sufficiently significant enough to be saved as "funny images" and hence 
represent noteworthiness sufficient to importance. 
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B. Picard and Hassan disclose claim 8, "The method of claim 7 wherein the user identifier 
and classification are Stored with the digital image in a digital image file" supra for claim 7. 

C. Picard and Hassan disclose claim 10, "The method of claim 7 wherein the step of 
classifying one or more of the images as an important image includes monitoring h facial 
expression of user" supra for claim 7, particularly at 'Wearcam [Man97) with affective sensing 
and pattern recognition' and ' funny images'. 

D. Picard and Hassan disclose claim 1 1 , "The method of claim 7 wherein the step of 
classifying one or more of the images as an important image includes monitoring the physiology 
of the user" supra for claim 7, particularly at These studies showed that heart rate variability 
was an indication of valence (whether or not the person found the photograph pleasant) and that 
the ratio of skin conductance to heart rate variability was an indicator of arousal.' 

E. Picard and Hassan disclose claim 12, "The method of claim 7 wherein the step of 
classifying one or more of the images as an important image includes providing a user interface 
to enable the user to indicate important images" supra for claim 7, wherein 6 WearCam [Man97]' 
corresponds to "user interface". 

6. Claim 9 is rejected under 35 U.S.C. 103(a) as being unpatentable over "Affective 
Wearables" by Picard, further in view of Hassan, as applied to claim 7 above, and further in view 
of Houvener et al., (US 5,832,464), hereinafter Houvener. 

A. Picard and Hassan disclose claim 9, "The method of claim 7 wherein the user identifier 
and the classification are stored in a database separate from the digital image" supra for claim 7. 
However, Picard and Hassan doe not appear to disclose, "wherein the user identifier and the 
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classification are stored in a database separate from the digital image", but Houvener does in col. 
4, Ins. 16-46 - see database with names and addresses augmented by a photographic image of 
each authorized user of a particular checking account. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply affective information to saving images disclosed by Picard in 
combination with supplemental ID stored with the digital image disclosed by Hassan, and 
motivated to combine the teachings because they both employ recording digital images with ID 
or labeling as revealed by Hassan in col. 1, lines 47-67 and by Picard using WearCam, and 
coupled with database disclosed by Houvener and motivated to couple the teachings because it 
provides a positive identity verification system and a secure and accurate database of 
photographic images of individuals and other pertinent data as revealed by Houvener in col. 3, 
Ins. 47-55. 



7. Claim 13 is rejected under 35 U.S.C. 103(a) as being unpatentable over "Affective 
Wearables" by Picard, further in view of Hassan, as applied to claim 7 above, and further in view 
of Dunn et aL, (US 6,154,772), hereinafter Dunn. 

A. Picard and Hassan disclose claim 13, "The method of claim 7 wherein the step of 
classifying one or more of the images as an important image includes determining the duration of 
time the user views each of the plurality of digital images" supra for claim 7. However, Picard 
and Hassan do not appear to disclose, "determining the duration of time the user views each of 
the plurality of digital images", but Dunn does in col. 21, Ins. 58-65, wherein 'collecting 
statistics on users' channel viewing habits (ie., which channels were viewed over a particular 
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time period)' corresponds to "determining the duration of time the user views each of the 
plurality of digital images". 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply affective information to saving images disclosed by Picard in 
combination with supplemental ID stored with the digital image disclosed by Hassan, and 
motivated to combine the teachings because they both employ recording digital images with ID 
or labeling as revealed by Hassan in col. 1, lines 47-67 and by Picard using WearCam, and 
coupled with collecting statistics on viewing habits (which channels were viewed over a 
particular time period) disclosed by Dunn and motivated to couple the teachings because it 
would let a user may select a program for viewing at any arbitrary time as revealed by Dunn in 
col 1,1ns. 41-42. 

8. Claim 14 is rejected under 35 U.S.C. 103(a) as being unpatentable over "Affective 
Wearables" by Picard, further in view of Hassan, as applied to claim 7 above, and further in view 
of Guedalia et al., (US 6,721,952 Bl), hereinafter Guedalia. 

A. Picard and Hassan disclose claim 14, "The method of claim 7 wherein the step of 
classifying one or more of the images as an important image includes monitoring the gaze of the 
user" supra for claim 7. However, Picard and Hassan do not appear to disclose, "includes 
monitoring the gaze of the user", but Guedalia does in col. 6, Ins. 45-52. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply affective information to saving images disclosed by Picard in 
combination with supplemental ID stored with the digital image disclosed by Hassan, and 



Application/Control Number: 1 0/036, 1 1 3 Page 8 

Art Unit: 2628 

motivated to combine the teachings because they both employ recording digital images with ID 
or labeling as revealed by Hassan in col. 1, lines 47-67 and by Picard using WearCam, and 
coupled with gazing at a large still image including a digital image generator providing a digital 
image as disclosed by Guedalia and motivated to couple the teachings because it would affect 
interactive gazing as revealed by Guedalia in col. 6, In. 52. 

9. Claims 15-20 are rejected under 35 U.S.C. 103(a) as being unpatentable over "Affective 
Wearables" by Picard, further in view of Hassan, as applied to claim 7 above, and further in view 
of Block Buster and/or Hollywood video rentals prior to 12/26/2001 exemplified by Rob Roy. 
A. Picard and Hassan disclose claim 15, "The method of claim 7 wherein the classifying 
step further specifies the time or period within a range of times that the classification was 
performed" supra for claim 7. However, Picard and Hassan do not appear, except for the time- 
stamp by Picard, to disclose, "wherein the classifying step further specifies the time or period 
within a range of times that the classification was performed", but Rob Roy does via [copyright 
date, 1995]. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply affective information to saving images disclosed by Picard in 
combination with supplemental ID stored with the digital image disclosed by Hassan, and 
motivated to combine the teachings because they both employ recording digital images with ID 
or labeling as revealed by Hassan in col. 1, lines 47-67 and by Picard using WearCam, and 
coupled with copyright date of collection of video images as disclosed by Rob Roy and 
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motivated to couple the teachings because it would be similar to the automatically saved time- 
stamp as revealed by Picard on p. 92, under 'Prototype of an Affective Wearable Computer 5 . 

B. Picard and Hassan disclose claim 1 6, "A method of using the user identifier and 
classification according to claim 7 to retrieve and display images, and further including the step 
of using the classification to determine the order or size of the displayed images" supra for claim 
7. However, Picard and Hassan do not appear to disclose, "further including the step of using the 
classification to determine the order or size of the displayed images", but Rob Roy does using 
[G, PG, PG-13, R and X ratings are grouped in order according to said rating classification and 
important popular new DVDs display kiosk size images like that on the cover of the Rob Roy 
DVD, while less prominent scenes are depicted smaller in size as illustrated on back of DVD 
cover where Rob Roy is hugging actress Jessica Lange.] 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply affective information to saving images disclosed by Picard in 
combination with supplemental ID stored with the digital image disclosed by Hassan, and 
motivated to combine the teachings because they both employ recording digital images with ID 
or labeling as revealed by Hassan in col. 1, lines 47-67 and by Picard using WearCam, and 
coupled with rating classification system and important popular DVDs displaying kiosk size 
images and less prominent scenes as disclosed by Rob Roy and motivated to couple the 
teachings because it would Rob Roy is simply a series of emotions string together with a plot as 
revealed by Picard under 4.3 Film/video. 

C. Picard and Hassan disclose claim 17, "method of claim 7 further including classifying 
and storing unimportant images" supra for claim 7. However, Picard and Hassan do not appear to 
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disclose, "further including classifying and storing unimportant images", but Rob Roy does [as 
exemplified by the smaller pictures on the back cover jacket of Rob Roy]. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply affective information to saving images disclosed by Picard in 
combination with supplemental ID stored with the digital image disclosed by Hassan, and 
motivated to combine the teachings because they both employ recording digital images with ID 
or labeling as revealed by Hassan in col. 1, lines 47-67 and by Picard using WearCam, and 
coupled with rating classification system and important popular DVDs displaying kiosk size 
images and less prominent scenes as disclosed by Rob Roy and motivated to couple the 
teachings because it would Rob Roy is simply a series of emotions string together with a plot as 
revealed by Picard under 4.3 Film/video. 

D. Picard and Hassan disclose claim 18, "The method of claim 7 further including providing 
user identifiers and classifications for a plurality of users" supra for claim 7. However, Picard 
and Hassan do not appear to disclose, "further including providing user identifiers and 
classifications for a plurality of users", but Rob Roy does [Drama, Comedy, Action, Adventure, 
Sci-Fi, and Horror; and/or G, PG, PG-13, R and X ratings identify the Drama, Comedy, Action, 
Adventure, Sci-Fi, and Horror; and/or G, PG, PG-13, R and X user(s)]. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply affective information to saving images disclosed by Picard in 
combination with supplemental ID stored with the digital image disclosed by Hassan, and 
motivated to combine the teachings because they both employ recording digital images with ID 
or labeling as revealed by Hassan in col. 1, lines 47-67 and by Picard using WearCam, and 



Application/Control Number: 1 0/036, 113 Page 1 1 

Art Unit: 2628 

coupled with rating classification system and important popular DVDs displaying kiosk size 
images and less prominent scenes as disclosed by Rob Roy and motivated to couple the 
teachings because it would Rob Roy is simply a series of emotions string together with a plot as 
revealed by Picard under 4.3 Film/video. 

E. Picard and Hassan disclose claim 19, "The method of claim 7 further including using the 
classification to determine the size of a displayed image" supra for claim 7. However, Picard and 
Hassan do not appear to disclose, "further including using the classification to determine the size 
of a displayed image", but Rob Roy does [wherein DVDs classified as new are displayed as 
kiosk size image posters]. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply affective information to saving images disclosed by Picard in 
combination with supplemental ID stored with the digital image disclosed by Hassan, and 
motivated to combine the teachings because they both employ recording digital images with ID 
or labeling as revealed by Hassan in col. 1, lines 47-67 and by Picard using WearCam, and 
coupled with rating classification system and important popular DVDs displaying kiosk size 
images and less prominent scenes as disclosed by Rob Roy and motivated to couple the 
teachings because it would Rob Roy is simply a series of emotions string together with a plot as 
revealed by Picard under 4.3 Film/video. 

F. Picard and Hassan disclose claim 20, "The method of claim 17 further including printing 
the important and unimportant images on at least one page and wherein the important images 
have a larger size than the unimportant images", but Rob Roy does [DVD, wherein DVD 
corresponds to a collection of digital images, i.e. album], and the images classified as important 
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images 'image of Rob Roy holding two pistols beside his wife' have a larger size than the 
images not classified as important images 'Scene Selections showing four images'", as [detailed 
supra]. 

Therefore it would have been obvious to one of ordinary skill in the art at the time the 
invention was made to apply affective information to saving images disclosed by Picard in 
combination with supplemental ID stored with the digital image disclosed by Hassan, and 
motivated to combine the teachings because they both employ recording digital images with ID 
or labeling as revealed by Hassan in col. 1, lines 47-67 and by Picard using WearCam, and 
coupled with rating classification system and important popular DVDs displaying kiosk size 
images and less prominent scenes as disclosed by Rob Roy and motivated to couple the 
teachings because it would Rob Roy is simply a series of emotions string together with a plot as 
revealed by Picard under 4.3 Film/video. 

Response to Arguments 

10. Applicant's arguments with respect to claims 7-20 have been considered but are moot in 
view of the new ground(s) of rejection. 

Conclusion 

1 1 . Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Responses 

12. Responses to this action should be mailed to: Commissioner of Patents and Trademarks, 
Washington, D.C. 20231. 

Inquiries 

13. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Gregory F. Cunningham whose telephone number is (571) 272- 
7784. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kee Tung can be reached on (571) 272-7794. The Central FAX Number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 




Gregory F. Cunningham 

Examiner 

Art Unit 2628 




